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§1. Introduction and results 

In the present paper we apply the theory of skew-symmetric vanishing lattices developed 
around 15 years ago by B. Wajnryb, S. Chmutov, and W. Janssen for the necessities of the 
singularity theory to the enumeration of connected components in the intersection of two 
open opposite Schubert cells in the space of complete real flags. Let us briefly recall the 
main topological problem considered in [SSV] and reduced there to a group-theoretical 
question solved below. Let N n+1 be the group of real unipotent uppertriangular (n + 1) x 
(n + 1) matrices and Di be the determinant of the submatrix formed by the first % rows 
and the last i columns. Denote by Aj the divisor {Di — 0} C N n+1 and let A n+1 be the 
union U" =1 Aj. Consider now the complement U n+1 = N n+1 \A n+1 . The space U n+1 can 
be interpreted as the intersection of two open opposite Schubert cells in S L n+ i(R) / B . 

In [SSV] we have studied the number of connected components in U n+1 . The main 
result of [SSV] can be stated as follows. 

Consider the vector space T n (F2) of upper triangular matrices with F2-valued entries. 
We define the group <& n as the subgroup of GL{T n (F2)) generated by F2-linear transfor- 
mations gij, 1 ^ % ^ j ^ n — 1. The generator gij acts on a matrix M G T n (F2) as follows. 
Let M 1 - 7 denote the 2x2 submatrix of M formed by rows % and % + 1 and columns j and 
j + 1 (or its upper triangle in case % = j). Then g^ applied to M changes M y by adding 
to each entry of M y the F2-valued trace of M lJ , and does not change all the other entries 
of M. For example, if i < j, then g^ changes M lJ as follows: 

rriij m iJ+1 \ ^ / m i+1 j + i m,j+l + ™>ij + m+l,j+l 

rrii+ij m i+1 j +1 J \ rrii+ij + my + m i+ i j +i rriij 

All the other entries of M are preserved. The above action on T n (F2) is called the first 
<3 n -action. 
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Proposition (Main Theorem of [SSV]). The number § n +i of connected components in 
l/n+i coinci^s w nfi fh e number of orbits of the first <S n -action. 

Here we calculate the number of orbits of the first n -action and prove the following 
result (conjectured in [SSV]). 

Main Theorem. The number fj n +i of connected components in U n+1 (or, equivalently, 
the number of orbits of the first <5 n - action) equals 3x2" for all n ^ 5. 

Cases n + 1 = 2, 3, 4 or 5 are exceptional and %i = 2, $3 = 6, $4 = 20, $5 = 52. 

The structure of the paper is as follows. In §2 we give a detailed description of the first 
<S n -action and its quotient by the subspace of invariants (called the second (J5 n -action). 
We formulate explicit conjectures about the types, number and cardinalities of orbits of 
both actions. In §3 we find linear invariants of these actions. In §4 we formulate a number 
of results about the monodromy group of a skew-symmetric vanishing lattice over F2. 
Using these results we count in §5 the number of orbits of the second (3 n -action. Finally, 
in §6 we explain the relation of the original (first) n -action to the monodromy group of 
the corresponding vanishing lattice and prove the results stated in §2. The concluding §7 
contains some final remarks and speculations about the origin and further applications of 
the above C5 n -action in Schubert calculus. 

We want to thank Andrei Zelevinsky for a number of stimulating discussions and his 
warm support of the whole project. We are indebted to Torsten Ekedahl for pointing out 
the importance of the dual representation of the group <5 n , to Anatoly Libgober for the key 
reference [Ja], and to Yakov Shlafman for writing sofisticated computer programms that 
enabled us to collect enough numerical data for putting forward the main conjecture. The 
third author expresses his gratitude to the Department of Mathematics, Royal Institute of 
Technology in Stockholm, for the hospitality and financial support during the final stage 
of the preparation of the manuscript. 

§2. TWO <S n - ACTIONS ON F 2 - VALUED UPPERTRIANGULAR MATRICES 
AND THEIR ORBITS. MAIN TABLES. 

2.1. The first (25 n -action and its invariants. Let G be a group acting on a linear 
space V over F 2 . We say that x G V is an invariant of the action if g(x) = x for any 
g G G, and that / G V* is a dual invariant if (g(x), f) = (x, f) for any x G V, g G G 
(here (•, •) is the standard coupling VxV*-> F 2 ). Evidently, dual invariants are just the 
invariants of the conjugate action of G on V*. 

In what follows we identify (T n (F 2 ))* with the space of F 2 -valued uppertriangular 
matrices in such a way that for any pair M G T n (F 2 ), M' G (T n (F 2 ))* one has (M, M') = 

Sl^j^j^n m ij m ij- 

Let us define the matrices Ri G (T n (F 2 ))*, 1 ^ i < n, as follows: all the entries of 
the rectangular submatrix of Ri formed by the first i rows and the last n + 1 — % columns 
are ones, and all the other entries of Ri are zeros. Thus, for any M G T n (F 2 ) the value 
(M, Ri) is just the sum of the entries of M mod 2 over the corresponding pattern pi (see 
Fig. 1). 
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Fig. 1. Patterns of dual invariants for the first <5 n - action 
IN CASE n = 8 



Let Ei G T n (F2), 1 ^ i ^ n, be the matrix whose ith diagonal contains only ones, and 
all the other entries are zeros. 

Theorem 2.1. (i) The subspace 1 n C T n (F2) of invariants of the first <3 n -action is an 
n- dimensional vector space. It has a basis consisting of the matrices E\, ... , E n . 

(ii) The subspace V n C (T n (F 2 ))* of dual invariants of the first (3 n -action is an n- 
dimensional vector space. It has a basis consisting of the matrices R±, . . . , R n . 

Let C T n {¥2) be the subspace orthogonal to V n with respect to the standard 
coupling. The translation of by an arbitrary element M G T n {¥2) we call a stratum of 
the first (25 n -action. By the definition, all the dual invariants are fixed at all the elements of 
a stratum. An n-dimensional vector h s — (/if, . . . , h%) is said to be the height of a stratum 
S (with respect to the basis {Ri}) if (M,Ri) = hf for any M G S. A stratum is called 
symmetric if its height is symmetric with respect to its middle, that is, if hf = hf_ i+1 for 
all 1 ^ i ^ n. 

Evidently, each stratum is a union of certain orbits of the first ® n -action. The structure 
of strata is described by Theorem 2.2 below. For the sake of simplicity, we omit the words 
"of the first n -action" in the formulation and write just "orbit" and "stratum" . 

Theorem 2.2. (i) Let n = 2k + 1 ^ 5, then 

each of2 k+1 symmetric strata consists of one orbit of length 2 2k + k ~ 1 —Sk2 k one 
orbit of length 2 2k + k ~ 1 + £ k 2 k + k ~ 1 — 2 fc , and 2 k orbits of length 1, where = —1 for 
k = At + 1 and 6^ = 1 otherwise; 

each of 2 n — 2 k+1 nonsymmetric strata consists of two orbits of length 2 2k 

(ii) Let n = 2k ^ 6, and let h denote the vector of length n the first k entries of which 
are equal to (1, 0, 1, 0, ...) and the last k entries vanish, then 

each of 2 k symmetric strata consists of two orbits of length (2 2fc ( fc_1 ) — l)2 fc_1 and 2 k 
orbits of length 1; 

each of2 k nonsymmetric strata S such that h s —h is symmetric with respect to its middle 
consists of one orbit of length (2 k ( k ~ 1 ^ — l)2 k _1 and one orbit of length (2 k ( k ~^ + l)2 k -1 ; 
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each of 2 n — 2 k+1 remaining nonsymmetric strata consists of two orbits of length 

22k 2 -k-l 

The assertions of Theorem 2.2 can be summarized in the following table. Observe that 
the total number of orbits in both cases equals 3 x 2", and we thus get Main Theorem. 

: n = 2k + 1 : n = 2k 



type : cardinality jj or b : cardinality (tor& 

trivial : 1 2 2k+1 : 1 2 2k 

standard : 2 2k2 + k ~ 1 2 2k + 2 - 2 k + 2 : 2 2fc2 - fc " 1 2 2k+1 - 2 k + 2 

type 1 : (2 fc2 - £ fc )2 fc2 + fc " 1 2 k+1 : 

type 2 : (2 fc2 - e k )(2 k2 - 1 + e k )2 k 2 k+1 : 

type 3 : : (2 2fc ( fc " 1 ) - l)2 fc " 1 2 k+1 

type 4 : : (2 k ^-^ - l)2 fc2 " 1 2 k 

type 5 : : (2 k{ - k ~^ + l)2 fc2 " 1 2 k 



Table 1. The orbits of the first (S n -action 

2.2. The second (5 n -action and its invariants. Let us introduce a (J5 n -action on 
T n_1 (F2) closely related to the first (J5 n -action on T n (F2), which we will call the second 
<3 n -action. This action is induced by taking the quotient modulo the subspace X n of the 
invariants of the first (3 n -action. Recall that by Theorem 2.1(i) J n is an n-dimensional 
subspace of all matrices having the same entry on each diagonal. One can suggest the 
following natural description of the second C5 n -action. 

Consider the linear map T n (¥ 2 ) -> T n-1 (F 2 ) such that the (i,j)th entry in the 
image equals the sum of the (i,j)th and the (i + l,j + l)th entries in the inverse image 
(thus entry in the image can be considered as representing the submatrix M lj of 

the initial matrix). Evidently, ker\l/ n = I n , and we obtain the induced action of (J5 n on 
T n_1 (F2). For any M G T n_1 (F2) the generator affects only the 3x3 submatrix of 
M centered at rriij by adding m^- to all entries marked by asterisks in the shape below: 

(* * 
* rriij * 
* * 

Here we use the convention that if the above 3 x 3-shape does not fit completely in the 
upper triangle, then we change only the entries that fit. 

For any i, 1 ^ % ^ k = [n/2], we define a subset 7Tj of the entries of the (n — l)x(n — 1) 
triangular shape as follows. The subset nk coincides with the initial (n — 1) x (n — 1) 
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triangular shape. To build 7r», 1 ^ i fc — 1, cut off the three i x i triangular shapes 
placed in all the three corners of the initial shape and get a hexagonal shape x\ with the 
sides of lengths i + 1 and n — 2i — 1. The shape xj is included into a nested family of 
ji = min{z + l,n — 2i — 1} hexagonal shapes x^; the shape x\ is obtained by peeling the 
external layer of x{~ 1 • The last shape x{ z degenerates to a triangle (upper if j j = i + 1 
and lower otherwise). The subset 7Ti consists of the three i x i triangular shapes as above 
and [ii/2] layers of the form xV \ Xi 3+ > 1 ^ J ^ [ii/2]; here we assume that = 0, 

and so for ji even the last layer consists of the whole triangle x 3 i ■ 

We now define the matrices Pi G (T 71_1 (F2))*, 1 ^ i ^ /c, as follows: all the entries 
of Pi that belong to iii are ones, and all the other entries of Pi are zeros. Thus, for any 
M e T n_1 (F2) the value (M, Pi) is just the sum of the entries of M mod 2 over the 
corresponding pattern Tii (see Fig. 2). 




Fig. 2. Patterns of dual invariants for the second <S n - action 

IN CASE n = 8 



Theorem 2.3. (i) The subspace of invariants of the second (5 n -action is trivial. 

(ii) The subspace 3 n _i C (T n_1 (F 2 ))* of dual invariants of the second <3 n -action is a 
k- dimensional vector space, k = [n/2]. It has a basis consisting of the matrices Pi, ...,Pk- 

It follows immediately from Theorem 2.3(h) that Pi = Pi +Pj_i, where Po = 0, is also 
a basis of D n -i. The strata of the second (& n -action and their heights (with respect to the 
basis {Pi}), which we denote rj, are defined in the same way as for the first (3 n -action. It 
is easy to see that \E f n takes any stratum of the first n -action to a stratum of the second 
(3 n -action. Denote by ip n \ F2 — > F 2 the mapping that takes h s to 77*™ The following 
statement gives an explicit description of t(j n . 

Theorem 2.4. The heights of the strata S and ^ n (S) with respect to the bases {Ri} and 
{Pi} satisfy the relations 

vf n(S) = hf + hf +1 + h s n _ l + h s n _ l+1 for l^Kk, 

^ = + hn-h+l- 

The structure of the strata of the second 0^-action is described in Theorem 2.5 below. 
For the sake of simplicity, we omit the words "of the second (3 n -action" in the formulation 
and write just "orbit" and "stratum" . 
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Theorem 2.5. (i) Let n = 2k + 1 ^ 5, then 

the stratum at height (0, . . . , 0) consists of one orbit of length 2 2k _1 — Sk2 k -1 , one 
orbit of length 2 2k ~ 1 +Ek2 k _1 — 1, and one orbit of length 1, where Sk = — 1 fork = 4t + l 
and Sk = 1 otherwise] 

each of the other 2 k — 1 strata is an orbit of length 2 2k . 

(ii) Let n = 2k ^ 6, and let fj denote the vector of length k the first k — 1 entries of 
which are equal to 1 and the last entry equals k mod 2, then 

the stratum at height (0, . . . , 0) consists of one orbit of length 2 2fc ( fc_1 ) — 1 and one orbit 
of length 1; 

the stratum at height fj consists of one orbit of length (2 fc ( fc_1 ) — l)2 fc ( fc-1 ) _1 and one 
orbit of length {2 k ^ k ~^ + i^^" 1 )" 1 ; 

each of the other 2 k — 2 strata is an orbit of length 2 2k ( k ~ 1 \ 

The assertions of Theorem 2.5 can be summarized in the following table. Observe that 
the total number of orbits in both cases equals 2 k + 2. 





n = 2k + 1 




: n = 2k 




type : 


cardinality 


$orb 


: cardinality 


tlorfe 


trivial : 


1 


1 


: 1 


1 : 


standard : 


2 2k 2 


2 k - 1 : 


: 2 2fe ( fc_1 ) 


2 k - 2 : 


type 1 


(2 fc2 - £fc )2 fc2 - 1 


1 


: 


: 


type 2 : 


: (2 k2 - efc )(2* a - 1 + £fc ) 


1 


: 


: 


type 3 








: ( ^ 2 2A: ( fc - 1 ) - 1) 


1 : 


type 4 








: /2 fc ( fc - 1 ) — i)2 fc ( fc-1 ) -1 


1 : 


type 5 : 








: ( 2 fc(fc-i) _|_ 1 ) 2 fc ( fc - 1 )- 1 


1 : 



Table 2. The orbits of the second (J5 n -action 

Remark. The map sends the orbits of the first n -action to the orbits of the second 
(5 n -action with the same name (i.e. trivial to trivial, standard to standard, etc). 

§3. Invariants and dual invariants of the (J5 n - actions 

In this section we prove Theorems 2.1, 2.3 and 2.4. 

3.1. The first (J5 n -action. Recall that we have identified (T n (F2))* with the space 
of F2-valued uppertriangular matrices in such a way that for any pair M e T n (F2), 
M> G (T»(F 2 ))* one has (M, M') = Ei <4<i< „ ™H<y 
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Lemma 3.1. The conjugate to the first <5 n -action is given by 

n i+i,j + m i,j+i + m i+l,j'+l if k = i, I = j, 
(9ijM') kl = { m' i+lij + m' l j+l + m'^ if k = i + 1, I = j + 1, 

z otherwise. 



Proof. Indeed, the action of gij on T n (F 2 )) affects only M y , while the action of <7y on 
(T n (F 2 ))* defined above affects only (M') lJ . Therefore, it suffices to consider the actions 
in the corresponding 4-dimensional spaces. The matrix of g^ in coordinates m^ , m^j+i, 











1\ 


1 


i 





1 


1 





1 


1 


Vi 








0/ 



hence its conjugate is exactly as asserted by the lemma. □ 

Now we are ready to prove Theorem 2.1. 

Proof of Theorem 2.1. (i) Follows immediately from the fact that T n is defined by the 
equations rtiij + nii+ij+i =0, l^i^j^n — 1. 

(ii) By Lemma 3.1, X> n is defined by the equations • - +1 + m^ +1 j+m' i+1 +1 = 0, 
l^i^j^n — 1 (with the same as above convention concerning the case when only 
a part of the submatrix fits into the uppertriangular shape). One can prove easily that 
these ("j 1 ) equations are linearly independent. Indeed, each of m^,-, 1 < j < n, enters 
exactly two equations, while m' n only one equation. It follows immediately that none 
of the equations involving ■ may participate in a nontrivial linear combination. The 
rest of the equations correspond to the same situation in dimension n — 1, so their linear 
independence follows by induction. We thus get dimV n = n. It remains to show that 
. . . , R n provide a basis for V n . Evidently, all these matrices are linearly independent 
and satisfy the equations defining V n . □ 

3.2. The second C5 n -action. For any matrix entry we define its neighbors as all the 
entries marked by asterisks in (1), or, more precisely, those of mi-ij-i, m,i-ij, m^j-i, 
niij+i, mj+ij, and raj+i^+i that fit in the uppertriangular shape. It is helpful to consider 
a graph $) n -i whose vertices are all the matrix entries and edges join each entry with its 
neighbors. It is easy to see that £j n _i is an equilateral triangle with the sides of length 
n — 2 on the triangular lattice (see Fig. 3). One can give the following description of the 
second <3 n -action in terms of f) n _i: gij acts on the space of F 2 -valued functions on $) n -i 
by adding the value at m^- to the values at all of its neighbors. 
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Fig. 3. The graph fj n _i in case n = 8 



Lemma 3.2. TTie conjugate to the second (5 n -action acts as follows: gij adds to the value 
at rriij the sum of the values at all its neighbors. 

Proof. Similarly to the proof of Lemma 3.1, it suffices to consider the action in the corre- 
sponding 7-dimensional subspaces. The details are straightforward. □ 

To prove Theorem 2.3, we need the following technical proposition. Consider the sub- 
space C (T n_1 (F 2 ))* of all linear forms invariant under the subgroup of (5 n gen- 
erated by {gij\ i ^ 2}; evidently, D n _i C T> 1 n _ 1 . Let u: T) 1 n _ 1 — > F^ -1 denote the 
projection on the first row, and let Sym p C F?J denote the space of all vectors symmetric 
with respect to their middle. 

Lemma 3.3. (i) The ith row of an arbitrary matrix M G £>^_i belongs to Sym n_I , 
1 < i < n - 1. 

(ii) dim'Dl l _ 1 = n — 1. 

(iii) The image of u coincides with Sym n_1 . 

Proof. For n = 3 an immediate check shows that D \ consists of the following four matrices: 

(° !)■ (° !)■ i)- 9- 

and so all the assertions of the lemma hold true. 

Consider now an arbitrary M G ID^-i- Let a = (ax, . . . ,a n -±), b = (b%, . . . , & n -2), 
c = (ci, . . . , c n -3) be the first three rows of M. It follows easily from the definition of 
D^-i th & t a, b, c satisfy equations a, = a\ + bi + bi + + Cj_i, 1 ^ i ^ n — 1, where 
bo = cq = 0. Evidently, the submatrix of M obtained by deleting the first row belongs 
to D\_ 2 . By induction, we may assume that the rows of this matrix are symmetric, 
that is, bi = b n -i-i, 1 ^ i ^ n — 2, and Cj = c n _2-i, 1 ^ z ^ n — 3. We thus have 
a n -i = ai + b\ + b n -i + b n -i-i + c n -i-i = at, 1 ^ i ^ n — 1, and hence the first row of M 
is symmetric as well. Moreover, it is clear that the solutions of the above equations form 
a 1-dimensional affine subspace, and hence dimS)*^ = dimD^_ 2 + 1 = n — 1. To get the 
third statement it suffices to notice that the first two rows define M uniquely, and that 
the dimension of the subspace spanned by these two rows is exactly n — 1. □ 

Proof of Theorem 2. 3. (i) Follows immediately from the definition. 
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(ii) Let us define the linear mapping v. — > F^ -1 by the following rule: v%{M) is 
the sum of the neighbors of nrtu. By definition, D n -i = ker v, so in order to find dimH) n _i 
it suffices to determine the image of v. As in the proof of Lemma 3.3, denote by a and 
b the first two rows of M. Clearly, = dj+i + a%-i + bi + , 1 ^ i ^ n — 1, where 
ao = a n = bo = 0. These equations define a linear mapping that takes the space Sym n_1 
of the first rows to the image of v. It is easy to see that the corank of this mapping is zero 
for n — 1 = 2k and one for n — 1 = 2k — 1; in other words, it equals [n/2] — [(n — l)/2]. 
Since dimSym n_ = [n/2], we get that the dimension of the image of v equals \{n — l)/2]. 
It follows immediately that dimS) n _i = n — 1 — [(n — l)/2] = [n/2]. 

The remaining assertion about the structure of dual invariants Pi is rather obvious. Its 
proof consists of the two simple checks: that Pi's are linearly independent and that any 
matrix entry has an even number of nonzero neighbors (shown by shaded areas on Fig. 2). 
Both are straightforward. □ 

Now we can prove Theorem 2.4. 

Proof of Theorem 2.4- It follows easily from Theorem 2.1 that the space D n (¥2) of di- 
agonal n x n matrices is transversal to the strata of the first action; moreover, \l/ n takes 
D n (¥ 2 ) to D n - 1 (¥ 2 ). Therefore it is enough to check the assertion of the theorem only 
for diagonal matrices. For M e D n (¥ 2 ) and M' = * n (M) Theorems 2.1 and 2.3 give 

(M ', Pi) = m' u + vn! n _ i n _ % + m' n _ i+l n _ i+l = 

(ma + mj+i^+i) + (m n _^ n _i + m n _i + i )n _i_|_i) = 

(M, Ri) + (M, R l+1 ) + (M, R n - t ) + (M, R n - l+1 ) 

for 1 < i < k. For n = 2k + 1 this equality holds also for i = k, but now k + 1 = n — k, 
hence the two middle terms vanish and thus (M f , Pk) = (M, Rk) + (M, R n -k+i)- Finally, 
forn = 2k we have (M', P k ) = m' kk = (M,R k ) + (M,R k+1 ) = (M, R k ) + (M, R n -k+i)- □ 

§4. Some results about skew-symmetric vanishing lattices 

In this section we quote and prove a number of results concerning the natural action of 
a group generated by transvections preserving a given skew-symmetric bilinear form on a 
vector space over F 2 . Our main reference is [Ja]. The relation to our original problem is 
explained in details in the next section and is based on the fact that the conjugate to the 
second lS» n -action is exactly of this kind. This allows us to describe completely its orbits, 
as well as the orbits of the second (3 n -action itself. 

4.1. Vanishing lattices and their monodromy groups. We assume that V is a 
vector space over ¥ 2 equipped with a skew-symmetric bilinear form (•, •) = (■, L(-)), where 
L is a linear map L: V — > V*. A quadratic function q associated with (•, •) is an arbitrary 
F 2 -valued function on V satisfying 

q(Xx + fiy) = X 2 q(x) + u 2 q(y) + \u(x, y). 

It is clear that a quadratic function completely determines the corresponding bilinear 
form. With any basis B of V we associate a unique quadratic function qs by requiring it 
to take value 1 on all elements of B. 



10 



Let /C = ker L be the kernel of (•, •), k = dim/C, and (ei, fi, e m , f m , gi, g K ) be a 
symplectic basis for (■, ■), that is, (x, y) = J2( x iV'i + Vi x 'i)i where x = Yl x i e i + 12 x ifi + 
J2 x i9i an d y = J2Ui e i + YliVifi + YlVidi- ^ the restriction g|x: vanishes, then one 
can define the ^4r/ invariant of g by Arf(g) = a ( e i) a (fi)i see e -S- [Pf]- For fixed 

dimensions of V and /C there exist, up to isomorphisms, at most three possibilities: (i) 
q(K) = 0, Arf(g) = 1; (ii) q(K) = 0, Arf(qr) = 0; (iii) g(/C) = F 2 (and so k > 1). 

In the first case one has = 2 2m+K " 1 + 2 m+K " 1 , Iq' 1 ^ = 2 2m+K " 1 - 2 m + K - 1 , 

in the second case one has = 2 2m + K " 1 - 2 m+K " 1 , |g _1 (0)| = 2 2m+K " 1 + 2 m+K ~ 1 , 

and in the third case one has |g -1 (l)| = |<? _1 (0)| = 2 2m+K_1 . 

For any 5 G V we define the symplectic transvection T$: V — > V by T$(x) = x — (x, 5)5. 
(Notice that T$ is an element of the group Sp" V of the automorphisms of (V, (•, •))). Given 
any subset A G V, we let Ta C Sp" V denote the subgroup generated by the transvections 
T s ,5eA. 

The main object of this section is a vanishing lattice, that is, a triple (V, (•,•), A) 
satisfying the following three conditions: (i) A is a l?A-orbit; (ii) A generates V; (iii) if 
rank V > 1, then there exist Si, 52 G A such that (Si, 82) = 1. The group Ta is called the 
monodromy group of the lattice. 

We say that a basis B of V is weakly distinguished if r# = Ta- In this case respects 
<7b, so, in particular, 55(5) = 1 for all 5 G A. Bases £? and B' are called equivalent if 
B' <zT B ■ B and £? C T B / ■ One can easily see that if B and B' are equivalent, then 
T B = T B >, qB = Qb', and if B is a weakly distinguished basis for (V, (•, •), A), then so is 
B'. 

Let B = (pi, ■■■,bd) be a basis in V, d = dimV. We define the graph gr(B) of B as 
follows: B is its vertex set, and bi is connected by an edge with bj if (hi, bj) = 1. It is easy 
to see that gr(B) is connected if B is weakly distinguished. 

A basis is called special if it is equivalent to a basis B = (bi, b^) such that for some 
k, 1 ^ k ^ d, we have (bi, bj) = iff i = j or i,j ^ A; + 1, and nonspecial otherwise. A 
vanishing lattice admitting a special (resp. nonspecial) weakly distinguished basis and its 
monodromy group are called special (resp. nonspecial). 

Nonspecial monodromy groups have an especially simple characterization (we are pri- 
marily interested in this case since for n ^ 5 the group <S n introduced in §1 can be 
interpreted as a nonspecial monodromy group, see §5 for details.) 

Theorem 4.1 ([Ja], Th. 3.8). Let (V, (•, •), A) be a vanishing lattice admitting a nonspe- 
cial weakly distinguished basis B. Then T B coincides with the subgroup 0^(q B ) of Sp* V 
consisting of all automorphisms that preserve q B ■ 

It follows that the classification of nonspecial vanishing lattices reduces to the classi- 
fication of the corresponding quadratic functions. We thus see that for given m and n 
such that dim V = 2m + k, dim/C = n there exist exactly three nonspecial vanishing lat- 
tices, depending on the values of q B (IC) and Ari(q B ). They are denoted by 0\(2m, k,¥ 2 ), 
0\(2m, k,¥ 2 ), and (9*(2m, «, F 2 ), and correspond to the cases Ari(q B ) = 1, Ari(q B ) = 0, 
and q B ()C) = F 2 , respectively (see [Ja, 4.2] for details). 

The following statement, which can be extracted easily from [Ja, §4], provides a suffi- 
cient condition for a vanishing lattice to be nonspecial. 
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Lemma 4.2. A vanishing lattice is nonspecial if it admits a weakly distinguished basis 
B such that gr(B) contains the standard Dynkin diagram of the Coxeter group Eq as an 
induced subgraph. 

4.2. Orbits of nonspecial monodromy groups and of the conjugate ac- 
tions. First of all, let us find the number of orbits of the monodromy group in the 
nonspecial case. 

Lemma 4.3. Let (V, (•, •), A) be a vanishing lattice admitting a nonspecial weakly distin- 
guished basis B . Then the number of orbits ofTs equals 2 K + 2, where k = dim/C. These 
orbits are the 2 K points of JC and the sets q , ^ 1 (0) \ /C and (/^(l) \ /C. 

Proof. Evidently, any group generated by transvections acts trivially on JC. Next, in the 
nonspecial case g^ 1 (l)\/C is an orbit of Tb by Theorem 3.5 of [Ja]. To prove that g^ 1 (0)\/C 
is an orbit as well, take an arbitrary pair u, v /C such that Qb(u) = Qb(v) = 0. It is easy 
to see that there exist u',v' ^ JC such that qsiu') = qB(v') = 1 and (u,u') = (v,v') = 1. 
Define V u = {w E V: (w,u) = (w,u') = 0}, T> v = {w £ V: (w,v) = (w,v') = 0}. 
Evidently, JC is a subspace of both V u and V v , and dimV u = dim"D„ = d — 2. Let q u 
and q v be the restrictions of qs to V u and T> v , respectively. If gs(/C) = F 2 , then the 
same is true for q u and q v , and so the quadratic spaces (T> u , q u ) and (V v , q v ) are isometric. 
Otherwise, if ?s(/C) = 0, the Arf invariants for the forms qs, q u , qv are defined. Moreover, 
Arf(q u ) = Arf(q v ) = 1 + Arf(gg), and hence (T> u , q u ) and (V v , q v ) are again isometric by 
the Arf theorem (see [Pf]). In both cases the isometry V u — > V v can be extended to an 
isometry of the entire V by letting u \— > v, u! \— > v' . By Theorem 4.1 this isometry belongs 
to T B . □ 

Let us consider now the conjugate to the action of the monodromy group of a vanishing 
lattice. We start from the following well-known result. 

Lemma 4.4. Let G be a finite group acting on a finite- dimensional space V over a finite 
field t. Then the number of orbits of the action equals the number of orbits of the conjugate 
action. 

Proof. The number of orbits of a finite group action on any finite set X can be calculated 
using the Frobenius formula (see e.g. [Ke]): 

■ 1 5>*i. 



\G 



geG 



where X 9 is the set of all (7-stable points. But the set V 9 of the stable points for the linear 
operator E g coincides with ker(E g — E), where E is the identity operator. Therefore, |V S | = 
|£|dimker(i? 9 -£)_ The equality dim ker (E 9 - E) = dimker(£ g - E)* = dimker(£* - E) 
proves the statement. □ 

Consider now the relation between the orbits of a nonspecial monodromy group Y and 
the orbits of the conjugate action. Choose some basis {b±, . . . ,b K } of /C and a K-tuple 
77 = {771, . . . ,r/ K } G F£. Denote by A n the affine subspace of V* of codimension k that 
consists of all elements x E V* such that (bj, x) = rjj, 1 ^ j ^ k. Evidently, A v is invariant 
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under the conjugate action of V for any rj G F£. Moreover, since (•, •) is skew-symmetric, 
we see that the image L(V) of the map L: V — > V* coincides with A . 

Our strategy is as follows. To study the structure of the conjugate action we use an 
additional construction. We introduce two subspaces Vi C V and V2 C V, which are 
transversal to /C (therefore dim Vi = dim V2 = 2m) and Vi + V2 = V. To each subspace 
Vi corresponds a subgroup C T (generated by transvections w.r.t. the elements in Vi). 
One can easily see that the set Ti U 1?2 generates V. We first study the orbits of Tj and of 
the corresponding conjugate action separately (which is fairly simple), and then describe 
their interaction. We start with the case of one subspace. 

Consider a subspace Vi C V transversal to /C. The restriction of the form (•, •) is 
nondegenerate on V\. Let l?i C Y be the subgroup generated by the tranvections w.r.t. 
elements of Vi . 

Lemma 4.5. Let Y be nonspecial. Then for any n G F£ the affine subspace A v consists 
of three orbits of the conjugate Yi-action. 

Proof. Observe first that for any n G F£ the intersection of A v and Vj 1 C V* contains 
exactly one element, which we denote by A\. Indeed, if {v±, . . . , i>2m} is an arbitrary 
basis of Vi, then AJ is the unique solution of the equations (6j, A\) = rjj, 1 ^ j ^ k, 
(v j ,A v 1 ) = 0,l^j^2m. 

Next, let L\ be the restriction of L to Vi, and L\ be the composition of L\ with the 
translation by Af[. We say that A\ is the shift corresponding to A v ; observe that A\ = 0, 
and thus L\ = L\. Evidently, L\ provides an affine isomorphism between Vi and A v . 
Recall that the conjugate action of Y (and hence, of Ti) preserves A v . Moreover, since 
Ti preserves V±, the element A^ is a fixed point of the conjugate to the Ti-action, as the 
unique annihilator of Vi in A v . Therefore, the diagram 

Vi > A* 1 
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Vi — ^ A** 

is commutative for any g\ G Ti. Since Y is nonspecial, the same is true for Y\ (by 
Lemma 4.2). It remains to apply Lemma 4.3 with k = 0. □ 

Let us now choose one more subspace V2 C V transversal to /C such that Vi + V2 = V 
and Ti U Y 2 generates Y. To study the orbits of the conjugate T-action we have to find 
which orbits of the conjugate Traction are glued together by the conjugate r 2 -action. 
Let us define an affine isomorphism I 71 : V\ — > V2 by I v = (L^) -1 o L^, where L 2 is the 
restriction of L to V2. 

Lemma 4.6. Let Y be nonspecial. 

(i) For t] = the following alternative holds: the linear space A consists of two orbits 
of the conjugate Y-action if there existu.v G Vi such that qsiu) = 0, qsiv) = qB(I°(u)) = 
qB(I°(v)) = 1, and of three orbits of the conjugate Y-action otherwise. 

(ii) For any 77 7^ the following alternative holds: the affine subspace A v consists of 
one orbit of the conjugate Y-action if there exist u, v G Vi such that Qs('u) = 0, qs{v) = 
qB(I v (u)) = qB(I v (v)) = 1, and of two orbits of the conjugate Y-action otherwise. 
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Proof. The first part of this lemma follows easily from Lemma 4.3 with k = together 
with Lemma 4.5. To prove the second part we notice first that /^(O) 7^ 0, and then we 
are done by the same reasons. □ 

§5. Orbits of the second <3 n -ACTiON 

In this section we prove Theorem 2.5 describing the orbits of the second (5 n -action 
on the space of upper-triangular (n — 1) x (n — 1) matrices over F 2 . Observe that the 
basic convention in this section is different from that of §4, namely, the conjugate to the 
second (J5 n -action is the action preserving a natural skew-symmetric form, and we study 
the second (5 n -action itself as the conjugate to its conjugate using Lemmas 4.4-4.6. 

5.1. The conjugate to the second (J5 n -action. In this section we assume that 
V = (T n - 1 (F 2 ))*. Let us introduce a bilinear form (•,•) on V by (M,N) = 
^(ii,jiU*ada) mi ^h n i2,h^ where (n, ji), (i 2 , h) runs over all pairs of neighbors in #„_i, 
see §3. Evidently, (■, ■) is skew-symmetric. 

Let B n _i be the standard basis of V consisting of matrix entries. Elements of -B n _i 
correspond bijectively to the vertices of S) n -±. Denote by Q the quadratic function gs„_ 1 
associated with (•, •). To find the value Q(M) for an arbitrary matrix M G V we represent 
M as M = gB i oiibi and define suppM = {hi G B n -\ : on = 1}. Let gr(M) denote 
the subgraph of £j n _i induced by the vertices corresponding to suppM. The following 
statement follows easily from the definitions. 

Lemma 5.1. The value Q(M) equals mod 2 the number of vertices of gr(M) plus the 
number of its edges. 

As a corollary, we obtain the values of Q on the dual invariants Pi, . . . , 

Corollary 5.2. (i) Let n = 2k + 1, then Q(?i) = for 1 ^ i < k. 

(ii) Let n = 2k, then Q(Pi) — 1 for 1 ^ i ^ k — 1, and Q(Pk) = k mod 2. 

Proof, (i) It follows from the description of the patterns 7^ that gr(Pj), 1 ^ i < k, consists 
of three copies of several disjoint cycles, and one more copy of S)i with / = \n— Si— 2| + 1, 
provided ji = min{i + 1, n — 2i — 1} is even. Therefore, by Lemma 5.1, we have Q(Pi) = 
3i(2i — 1) mod 2 for ji = i + 1, i even, and Q(Pi) = 3i(2i — 1) +1(21 — 1) mod 2 otherwise. 
Since n is odd, we have / = i mod 2, and hence Q(Pi) = 0, 1 ^ z < k. Finally, for Pk 
we have Q(Pk) = (n — l)(2n — 3) = 2/c(4/c — 1) mod 2 = 0. It remains to notice that 

Q(Pi) = Q(Pi) + Q(Pi-i). 

(ii) In the same way as in (i) we see that if 1 ^ i < /c,then Q(Pi) = 3i(2i — 1) + 1(21 — 1) 
mod 2 for ji = i + 1, i odd, and Q(Pi) = 3i(2i — 1) mod 2 otherwise. Since n is even, we 
have l = i + l mod 2, and hence Q(Pi) = for even i < k and Q(Pi) = 1 for odd i < k. 
Finally, for P fc we have Q(P fc ) = (n- l)(2n-3) = (2k - l)(4fc - 3) mod 2 = 1. As in (i), 
the result follows from = Q(P) + <5(P_i). □ 

Let /C be the kernel of (•,•). Evidently, /C is the space of dual invariants of the second 
<8 n -action, hence Theorem 2.3 gives an explicit description of /C. Applying Corollary 5.2 
we get the following statement. 
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Corollary 5.3. Let n = 2k, then Q(IC) = F 2 and |Q~ 1 (0)| = = 2 2fc2 " fc " 1 , 

|Q-i(o) n K\ = |<5 _1 (i) n/Cj = 2 fc " 1 . 

Proof. It follows immediately from Corollary 5.2(ii) that Q(JC) = F 2 . The statement 
concerning the sizes of |Q _1 (0)| and |<5 _1 (1)| follows easily from the general description 
(see §4.1) with m = k(k — 1) and k = k. The last statement follows from the fact that 
the normal form of Q\jc in case Q{K) = F 2 is x\ (see [Pf]). □ 

In case n = 2k + 1 the situation is far more complicated. 

Lemma 5.4. Let n = 2k + 1, then Q(K) =0. // k = At + 1, i/ien Arf(Q) = 1 and 
|Q- 1 (0)| = 2 2fc2 + fc " 1 - 2 fc2+fc " 1 , = 2 2fc2+fc " 1 + 2 fc2+fc " 1 . Otherwise Arf(Q) = 

and IQ-^O)! = 2 2fc2+fc " 1 + 2 fc2 + fc "\ IQ" 1 ^)! = 2 2fc2+fc " 1 - 2 fc2+fc " 1 . 

Proof. It follows immediately from Corollary 5.2(i) that Q(JC) = 0, and hence the Arf 
invariant exists. Observe that by general theory of quadratic spaces (see §4.1) this means 
that |Q _1 (0)| ^ |<5 _1 (1)|- Moreover, IQ" 1 ^)! > |Q _1 (1)| implies Arf(Q) = 0, while 
|<2 -1 (0)| < |<5 _1 (1)| implies Arf(Q) = 1. Therefore, to find Arf(Q) it suffices to count the 
number of elements in V on which Q vanishes. 

Let uj: V — > F^ -1 denote the projection on the first row (see Lemma 3.3), and let 
V a = w _1 (a), a G FJT 1 . We say that V a is inessential if |Q~ 1 (0) n V a \ = |<2 _1 (1) n V a \, 
and essential otherwise. 

Let us choose M a e V a such that Q(M a ) = 0; if gr(a) contains an even number of 
connected components, one can take supp M a = supp a, otherwise supp M a contains one 
more vertex, which corresponds to the matrix entry (n, n). We thus have M a = M® + M^, 
where suppM^ = supp a and | suppM^| ^ 1. 

Let us define a function Q a on Vo by Q a (M) = Q(M + M a ) for any M e Vo; observe 
that Q a is just a shift of the restriction Q\ Va . Evidently, Q a (M) = Q(M) + (M,M a ). 
Therefore, Q a (M + N) - Q a (M) - Q a (N) = (M, N), which means that Q a and Q define 
the same bilinear form on Vo (observe that Vo is identified naturally with (T n_2 (F 2 ))*). 
Let us evaluate Q a on the kernel /Co; Qa(K-o) = F 2 would mean that Q vanishes exactly 
on a half of the elements of V a , in other words, that V a is inessential, while Q a (ICo) = 
would mean that V a is essential. 

Since both Q and Q a define the same bilinear form on Vo, we see that /Co is the space 
of dual invariants of the second l£» n _i-action; therefore, by Theorem 2.3, it has a basis 
{Pi, ... , K}- By Corollary 5.2(h), Q a (P z ) = 1 + (P, M a ) = l + (P, M a °) for 1 < i < k - 1 
and Qa(Pk) = k + (Pfc,M°). Therefore, V a is essential if and only if the entries of a 
satisfy equations ak-i + ctk+i+i = k + i mod 2 for ^ i ^ k — 1. It is easy to see that 
any solution of the above equations is represented as a = h + s, where h is as defined in 
Theorem 2.2(h) and s G Sym n_1 . It follows from Theorem 2.3 that for any s G Sym n_1 
there exists S G K, such that u(S) = s. Observe that in this case M i— > M + S takes 
V a to V a+S . Moreover, Q(M + M a + S) = Q(M + M a ), since (M + M a , S) = follows 
from S G K. and Q(S) = by Corollary 5.2(i). Therefore, all the essential subspaces V a 
influence Arf(Q) in the same way, hence, Arf(Q; V) = Arf(Q^; Vo). 

To study Arf (Q^) on Vo we reiterate the same process once more, that is, we decompose 
Vo into affine subspaces Vo&, where b is defined by the projection ujq of Vo on the first row. 
We choose M 0o G Vot similarly to M a and define a function Qob on Voo by Qob(M) = 
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Qh(M + M ob ). As before, V o is identified naturally with (T n_3 (F 2 ))*, and Q ob and 
Q define the same bilinear form on Voo- The corresponding kernel /Coo is the space 
of dual invariants of the second (5 n _2-action; its basis is {Pi, . . . , P^-i}- We thus get 
that Vob is essential if and only if the entries of b satisfy equations b^-i + bk+i = for 
1 ^ i ^ k — 1. Any solution of these equations belongs to Sym n_2 . As before, MkM + S 
with S e /C and u (S) = s takes V ob to V ,b +S and Q~ h (M + M ob + S) = Q~ h {M + M ob ); 
identity Q h (S) = follows readily from Q(Pi) = (P^M?), 1 ^ i ^ k - 1. We thus get 
Arf(^;Vo) = Arf(^;Voo). 

Recall that on V o one has Q~ h (M) = Q(M) + (M, M±). It is easy to see that for k = At 
and k = At + 3 one has = 0, and hence = Q on Voo- This means that the Arf 
invariant is constant on each triple of the form k = At + 2, At + 3, At + A. 

Let now k = At + 1 or k = At + 2, and hence ^ 0. We decompose Voo into four affine 

subspaces Vqq, Vqo, Voo, an d V(x). ^ ne su bspace Vqo consists of the matrices having i at 
position (n — l,n — 1) and j at position (to — 1, to). It is easy to see that = Q on Vqq; 
moreover, the involution M i— > M + reverses the value of Q (and thus of Q^) on these 
subspaces. Therefore, the subspaces Vq are inessential. On the other hand, = Q + 1 
on Vqq, i ^ j, and the involution M i— > M + in this case preserves the value of Q 
(and thus of Q^). Therefore, Arf(Q^; V o) = 1 + Arf(Q; Voo)- This means that the Arf 
invariant reverses twice on each triple of the form k = At, At + 1, At + 2. 

To complete the proof it is enough to check the value of the Arf invariant for k = 1. 
The exact values for the lengths of orbits follow easily from the general description (see 
§4.1) with m = k 2 and k = k. □ 

Let us now find the relation between the conjugate to the second (5 n -action and the 
theory of skew-symmetric vanishing lattices explained in §4. 

Lemma 5.5. (i) The triple (V, (■,-), B n _i) is a vanishing lattice. 

(ii) The conjugate to the second (3 n -action coincides with the action ofTB n _ 1 - 

(hi) The basis -B n -i is weakly distinguished and its graph gr(5 n _i) coincides with S) n -\. 

Proof, (i) We have to check that -B n _i satisfies conditions (i)-(iii) of the definition of the 
vanishing lattices. Condition (ii) is evident. To check to it suffices to take for 8\ and 82 
any pair of adjacent vertices of 5) n -i- Finally, to check (i) we take an arbitrary pair b, b' 
of adjacent vertices of £j n _i and find that T b T b i{b) = b' . 

(ii) Follows immediately from Lemma 3.2. 

(iii) Obvious. □ 

Lemma 5.6. The conjugate to the second (3 n -action is the action of a nonspecial mon- 
odromy group for to ^ 5. 

Proof. By Lemmas 4.2 and 5.5 we have to check that the Dynkin diagram of Eq is an 
induced subgraph of S) m for m ^ 4. Since S) m is an induced subgraph of fti for m < /, it 
suffices to find an induced subgraph corresponding to E G in 534, see Fig. 4. □ 
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Fig. 4. The graph fj 4 and the Dynkin diagram for E 6 



5.2. The orbits of the second (J5 n -action. First of all, let us find the number of orbits 
of the second n -action. 

Lemma 5.7. The number of orbits of the second <3 n -action equals 2 k + 2 for n ^ 5. 

Proof. Indeed, by Lemma 4.4 the number of orbits of the second (S^-action equals that 
of its conjugate. By Lemma 5.6 the conjugate to the second n -action is the action of a 
nonspecial monodromy group; hence, by Lemma 4.3 the number of its orbits equals 2 K + 2, 
where k is the dimension of /C. Since /C is just the space of dual invariants of the second 
(J5 n -action, Theorem 2.3(h) implies k = k. □ 

Let us now define two subsets of the vertex set of S) n -\ as follows: Ei consists of the 
vertices corresponding to the matrix entries (i,n — 1), 1 ^ i ^ k, and £2 of the vertices 
corresponding to (n — i,n — i), 1 ^ k, see Fig. 5. 



Following the strategy described in §4, we choose the following two subspaces in V = 
(T"- 1 (F 2 ))*: 



Let K, denote the kernel of (•, •). It follows easily from the explicit description of /C (see 
Theorem 2.3(h)) that codimVi = codimV2 = dim/C and Vi R /C = V2 H /C = 0, hence V± 
and V2 as above satisfy the assumptions of §4. 




Fig. 5. The sets Ei, E 2 , E 3 for the case n = 8 



Vi 

v 2 



{M e V: suppMH Ei = 0}, 
{MeV: suppMnE 2 = 0}. 
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Fix the basis Pi, P k of IC. It is easy to see that the stratum of the second (J5 n -action 
at height r\ is just the affine subspace A 71 as defined in §4.2. Therefore, the statements 
of Theorem 2.5(i) concerning the number and the lengths of orbits are yielded by the 
following proposition. 

Lemma 5.8. Let n = 2k + 1 ^ 5. 

(i) The linear subspace A C V* consists of three orbits of the second <3 n -action. The 
lengths of the orbits are 2 2k _1 -e k 2 k _1 , 2 2k _1 + e k 2 k _1 - 1, and 1, where e k = -1 for 
k = At + 1 and e k = 1 otherwise. 

(ii) For any nonzero r\ G ¥ k the affine subspace A v C V* is an orbit of the second 
<3 n -action. 

Proof, (i) Let us first find the number of orbits contained in .4°. By Lemma 4.6(i) it 
suffices to prove that Q(M) = Q(I°(M)) for any M G Vi. Define M K = M + 7°(M), 
and let L: V — > V* be the linear mapping associated with (•,•). Since by definition 
L(M) = L(I°(M)) for any M G V u we have L(M K ) = 0, and thus M K G IC. Therefore 
Q(I°(M)) = Q(M) + Q{M K ), and since by Lemma 5.4 Q{K) = 0, we are done. 

It follows now from Lemmas 4.5 and 4.3 that the three orbits in question are isomorphic 
to {0}, (Q- 1 (0)nVi)\{0}andg- 1 (l)nVi. Evidently, Arf(Q) = Arf(Q| Vl ) and (Vi,Q| Vl ) 
is a nondegenerate quadratic space of dimension 2k 2 . Therefore the statement concerning 
the lengths of the orbits follows from Lemma 5.4 and the general description of quadratic 
spaces (see §4.1) with m = k 2 and k = 0. 

(ii) By Lemma 5.7 the total number of orbits is 2 k + 2, and by part (i) of the present 
Lemma exactly three orbits are contained in .4°. Therefore, each of the 2 k — 1 remaining 
orbits coincides with the corresponding A v . □ 

The corresponding statements of Theorem 2.5(h) are yielded by the following proposi- 
tion. 

Lemma 5.9. Let n = 2k ^ 6 and fj = (Q(Pi), . . . , Q{Pk))- 

(i) The affine subspace A^ C V* consists of two orbits of the second <5 n -action. The 
lengths of the orbits are 2 2fc ( fc " 1 )" 1 - 2 fc ( fc " 1 )- 1 and 2 2fc ( fc " 1 )" 1 + 2 fc ^- 1 )" 1 . 

(ii) The linear subspace A C V* consists of two orbits of the second (3 n -action. The 
lengths of the orbits are 2 2fc ( fc_1 ) — 1 and 1. 

(iii) For any rj e F§, rj ^ 0,fj, the affine subspace A v C V* is an orbit of the second 
<3 n -action. 

Proof, (i) Let Af[ e V* be the shift corresponding to Vi, as defined in the proof of 
Lemma 4.5, and A% be the similar shift corresponding to Vi- Observe that + A% G .4°, 
and hence there exists HJJ G V2 such that L^H^) = A\ + A%. 

For any M G Vi put M K = M + I T1 (M) +H%. It follows from the definition of I v that 
L(I^{M)) = L(M) +A V 1 + A\, hence L(M K ) = 0, that is, M K G IC. 

Let us now find Q(I^(M)). Since 7"(M) = M+H^ + M K and (M, H%) = (M, A^ + A^), 
one has Q(7"(M)) = Q(M) + Q(H^)+Q(M !C ) + (M, A\ + Al), where (•, •) is the standard 
coupling between V and V* defined in §2. 

Since M G Vi, we get (M, Af[ + A%) = (M, A%) = \ supp M n supp It follows easily 
from Theorem 2.3(h) that supp A?? = {(n — i,n — i) G E 2 : Q(Pi) = 1}- We thus have 
(M,AJ + ^) = |{(n-z,n-z) G E 2 nsuppM: = 1}|. 
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On the other hand, M K = J2i p i : 

(n — i,n — i) G E 2 fl supp M^}, and hence Q(M K ) = 
\{(n-i,n-i) G E 2 f]snppM !C : Q(P;) = Finally, M + M K = I V (M) + G V 2 , and 
hence E 2 H supp M = E 2 n supp M K . 

Thus, Q(I f '(M)) — Q(M) = <5(-ff 2 ) does not depend on M, and hence u, v as in 
Lemma 4.6(h) do not exist. Therefore, A** consists of the two orbits of the second in- 
action. For the same reason the trivial one-element orbit obtained from {0} is glued to 
the image of Q -1 (0) \ {0}, and hence the lengths of the orbits are the sizes of <5 _1 (0) 
and <5 _1 (1) for the nondegenerate case. Thus k = and m = k(k — 1), which yield the 
required result. 

(ii), (iii) Follows immediately from Lemma 5.7 and part (i) in the same way as in 
Lemma 5.8(h). To find the lengths of the orbits in A observe that Za(0) = L 2 (0) = G 
A , hence the two parts that are glued together are Q _1 (0) \ {0} and <5 _1 (1), and the 
result follows. □ 

To obtain Theorem 2.5(h) it suffices to notice that by Corollary 5.2(h) f\ as defined in 
Lemma 5.9 coincides with fj as defined in Theorem 2.5(h). 

§6. Orbits of the first <5 n -ACTioN 

In this section we prove Theorem 2.2 concerning the structure of orbits of the first 
G n -action. 

6.1. The structure of the strata of the first S5 n -action. Let us introduce some 
notation. We write X ~ X if the matrices X and X belong to the same orbit. We do not 
specify in this notation which (3 n -action is meant; this should be always clear from the 
context, since each space under consideration carries exactly one (3 n -action. 

Let O = UJ1U2 ■ ■ - u>t denote an arbitrary word in alphabet {gij}', we write O for the 
word uit ■ ■ ■ u>2U>i ■ For any subset E of the alphabet we define [O : E] as the num- 
ber of occurencies of g a , a G E, in O. We write QX to denote the matrix obtained 
from X by applying to it the elements u>i,u>2, . . . ,u t in this order. We say that Q, is 
X-nonredundant if {uii . . .u s }X 7^ {uji . . . uj s -i}X for s = l,...,t (here lvqX = X). 
Evidently, X-nonredundancy is preserved under isomorphisms and under \l/ n . 

It is easy to see that each stratum S h of the first (J5 n -action is a covering of degree 2 k 
over the stratum ty n (S h ) of the second (25 n -action. We are going to find a family of linear 

functionals C h : T n (F 2 ) — > F 2 with the following property: let M, M G S be an arbitrary 
pair of matrices such that * n (M) = * n (M), then M ~ M if and only if C h (M) = C h (M). 
The existence of such a family would imply immediately that each nontrivial orbit of the 
first (3 n -action is a covering over the corresponding orbit of the second n -action of degree 
2 fc_1 , except for the cases when C h is trivial; in the latter case the degree equals 2 fc . 

For any M G T n (F 2 ) we denote by S(M) the set of matrices M such that * n (M) = 
\l/ n (M) and M,M belong to the same stratum of the first C5 n -action. Further, let W 
denote the /c-dimensional subspace of T n (F 2 ) generated by the entries (l,i), 1 ^ % ^ k, 
and r: T n (F 2 ) — > W denote the natural projection. 

Lemma 6.1. The projection t provides an affine isomorphism between S(M) and W for 
any M G T n (F 2 ). 
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Proof. Indeed, S(M) =M + (ker # n n T>£) = M + (Z n n Z^). It follows from the explicit 
description of V n (see Theorem 2.1(h)) that dim(X n n P„) = fc and ker r|x ran £>x =0. □ 

Let us introduce, in addition to Ei and E 2 , one more subset of the vertex set of £) n _i: 
E3, consisting of vertices corresponding to matrix entries (1, z), 1 ^ i ^ k, see Fig. 5. As 
in §5.2, we define the subspace V 3 C V = (T n - 1 (F 2 ))*, 

V 3 = {MgV: suppMnE 3 = 0}. 

Recall that L provides an isomorphism between V3 and A = £>„_i, and A\ + A\ G ^4°; 
therefore, there exists H^ G V3 such that L(H^) = A\ + A%. 

Lemma 6.2. Q(-H^) + (A??, H%) = /or any rj G F§. 

Proof. Let us give an explicit description of ifg . We define the matrices Hi G V, 1 ^ i ^ fc, 
in the following way. Let 1 ^ 2 < /c, then supp Hi is obtained from the initial shape by 
deleting the upper-left justified (n — i — 1) x (n — i — 1) triangle and upper-right justified 
(i — 1) x (i — 1) triangle. For n = 2k + 1 we define in the same way, and for n — 2k 
suppiffc is the lower-right justified {k — 1) x (k — 1) triangle, see Fig. 6. 



supp //] 



supp H : 



supp H 



supp H 4 



Fig. 6. Supports of the matrices Hi in case n = 8 



Evidently, Hi G V3, 1 ^ i ^ k. Besides, Q(Hi) = 1 for 1 ^ i < k and Q(Hk) = n 
mod 2. Finally, (iifj, Hj) = 1 for i ^ j. 

Observe now that ifg = XX ^ ■ {n — i,n — i) G supp A^}. Therefore, for n = 2/c + 1 
we have Q(H^) = c + c(c — l)/2 = c(c + l)/2 mod 2, where c = | supp A%\. On the other 
hand, {Al,H%) = 1 + 2 + •■■ + c = c(c + l)/2 mod 2. Hence Q(H%) + (A%, H%) =c(c + l) 
mod 2 = 0. 

Let now n = 2/c. If (n — k,n — k) ^ supp A\, then the above proof applies. Otherwise 
Q(H%) = c-l+c(c-l)/2 = (c-l)(c+2)/2 mod 2 and (A%, H%) = 1+2+- • -+c-l+c-l = 
(c-l)(c + 2)/2 mod 2, and hence Q{H%) + (A%,H%) = (c - l)(c + 2) mod 2 = 0. □ 

Let us define a family of functionals f^-.JC^ F 2 , 77 G F^, as follows: f v (X) = Q(X) + 
(A^X). It is easy to see that f v is a linear functional for any 77, since Q is linear on /C. 
The following statement stems immediately from Corollary 5.2. 

Lemma 6.3. (i) Let n = 2k + 1, then f v is nontrivial if and only 
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(ii) Let n = 2k, then f v is nontrivial if and only if rj 7^ rj = (1, . . . , 1, k mod 2). 

Let now 9: V — > /C denote the projection on K, along V3. We lift f n to the whole V 
with the help of 9 by defining F^{X) = f v (9(X)) for any X G V. 

It is easy to check that the standard coupling (•, •) restricted to £3 identifies /C* with 
V3 1 C V*. Let p = (/f, . . . , f%) G V3-; we define a linear functional = (c^ , . . . , c£) G 
W* as follows: c h = A/^W, where A is the matrix in GL k (¥ 2 ) such that Ay = 1 iff 
% «C j, and lift it to the whole T n (F 2 ) with the help of r by defining C h {M) = c h (r(M)) 
for any M G T n (F 2 ). 

Lemma 6.4. Let M G T n (F 2 ) fee an arbitrary matrix in S h , h G F^. If M E S(M) and 
M ' ~ M, then C h (M) = C h (M) . 

Proo/. Indeed, let 7/ = V> n (/i) and let M' = gij M, M' ^ M. Then C h (M') = C h (M) if 
(i, j) £ £3 and C h (M') - C h (M) = c) + cj +1 = f] otherwise. Hence the total variation 
of C h (M) along the trajectory defined by an arbitrary M-nonredundant word O equals 
var^ C = [O : suppF^] mod 2. Therefore, to prove the Lemma it suffices to show that 
for any M-nonredundant word Q such that M = QM the number [O : supp F 11 ] is even. 

Evidently, there exists a decomposition fi = fiifi 2 . . . fi r with Oj 7^ for % ^ 2 such that 
all the elements of 2 j +1 belong to the subgroup (3* C <3 n generated by the elements of 
Vi, and all the elements of 2i belong to the subgroup (3^ C <3 n generated by the elements 
of V 2 . By Lemma 4.5, instead of looking at the trajectory of M 2 i = {fii . . . fi 2 i}\I/ n (M) 
under 2 i_|_i one can look at the trajectory of (L^) _1 (M 2 i) in Vi, and instead of the 
trajectory of M 2 i+i = Vt 2 i + \M 2 i under 2 i +2 , at the trajectory of (L^) -1 (M 2 i+i) in V 2 . 
These pieces are glued together with the help of the isomorphism I v , see Fig. 7. Let us 
find the total variation var^ F v of the functional F v along the obtained trajectory. 




Fig. 7. Trajectory defined by O in the charts Vi and V 2 

It follows immediately from Lemma 3.2 that for any X £ V and X' = gijX 7^ X one 
has F 7 ? (X') - F^(X) = 1 iff (i, j) G suppF 7 ?. Therefore, the total variation of F v along 
the pieces of the trajectory that lie entirely in V\ or V 2 equals [Q : suppF 77 ] mod 2. 

Let us find the variation of F v under the isomorphism I v . Similarly to the proof of 
Lemma 5.9, for any X G Vi one has I^{X) = X + H% + X K with X K G K. Thus, 
- FV{X) = FV{H%) + F r, (X lc ) = Q{X K ) + (A%, X K ), since 6(H$) = 0. 
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On the other hand, Q(I V (X)) - Q(X) = Q{X K ) + Q(H%) + {A\ + A%,X). Besides, 
(A%,X) = since X E Vi, (A^X + H^ + X^) = since /"(X)) = X + Hg + X* E V 2 , and 
Q(H%) + (A%, H%) = by Lemma 6.2. Adding the last four equalities we get Q(I V (X)) - 
Q(X) = Q{X K ) + (A%, X K ). Therefore, i™ - F^{X) = Q{I^{X)) - Q{X). 

Recall that by Lemma 4.3 Q is constant on each orbit in Vi and V 2 . Therefore, 
varnF 7 ' = var n Q + [O : suppF 77 ] mod 2. Since * n (M) = *„(M), the trajectory de- 
fined by O is a loop, and hence var^ F v = var^ (5 = 0, thus var^ C 71 = [O : supp F 7 ?] 
mod 2 = 0. □ 

Let us now prove the converse statement. 

Lemma Ji.5. Let M E T^F^) \X n be an arbitrary matrix in S h , heW%. If M E S(M) 
and C h (M) = C h (M), then M ~ M. 

Proof. In what follows we assume that h is fixed and rj = ift n (h). First of all, for any 
Z E /C we choose two matrices Mq , M z E V\ such that 

Q{I\Mf))-Q{Mf) = Z, Q(M?) = i, i = 0,l. 

Such a pair exists for any Z E /C, since the first condition defines a translation of V\ D V 2 , 
and Q is nonlinear on Vi fl V 2 , and hence nonconstant on any translation of Vi D V 2 . 
Moreover, we can further assume that ^ and ^(Mf) ^ for z = 0, 1. 

Let us fix an arbitrary nontrivial matrix X E V\. By Lemma 4.3, X, X' = M^ x y 
and X" = Mq^ x ^ belong to the same 0^-orbit. Let Q' z be an X-nonredundant word in 
{gij}, (i,j) £ Ei, such that X' = tt' z X. We denote Y' = I^X'), Y" = I^(X"). By the 
proof of Lemma 6.4, the variation of Q under I v equals the variation of F v . Therefore, 
if p(Z) = 0, then Q(Y') = Q(X') = Q(X) = Q(X") = Q(Y"), and hence Y',Y" E V 2 
belong to the same 0^-orbit. Let Q' z denote a y'-nonredundant word in {gij}, £ E 2 , 
such that Y" = Q,' Z Y' . Evidently, O z = Q.' Z Q. Z Q.' is X-nonredundant and fl z X = X. 

By Lemma 6.1, it suffices to prove that the value of any linear functional other than 
c h can be changed along an appropriate Vtz with f r, (Z) = 0. Take any W E /C, and let 
f w E V3- be the conjugate to W with respect to (•, •) restricted to E 3 ; put F w = f w o 9. 
Then, similarly to the proof of Lemma 6.4, = var nz F w = [fl z : suppF^] + f w (Z) 
mod 2. Therefore, for c w = Kf w and C w = c w or one has var^ z C w = [ftz '■ suppF w ] 
mod 2 = f w {Z). 

Let now W = X)i=i Evidently, it is enough to consider the following two types of 

W: (i) u>i = for i 7^ j, Wj = 1, where rjj = 0, and (ii) u>i = for i 7^ 31,32, w j! = w j 2 = 1) 
where r]j 1 = rjj 2 = 1. In order to get varn z C w = 1 for n = 2k + 1 one takes Z = Pj in the 
first case and Z = Pj 1 + Pj 3 with rjj 3 = 1 in the second case. For n = 2k one takes either 
Z = Pj + Pj 3 with i]j 3 = or Z = Pj 3 with i]j 3 = 1 in the first case and Z = Pj x + Pj 3 
with i]j 3 = 1 in the second case. □ 

Proof of Theorem 2.2. Let n = 2k + 1 and ^ n {h) 7^ 0. Then by Theorem 2.5 and 
Lemmas 6.3, 6.4, 6.5 the stratum S h consists of two orbits distinguished by C . Their 
lengths are equal since C h is linear and nontrivial. For ip n (h) = Lemmas 6.3 and 6.5 
imply that nontrivial orbits in S h are just coverings of degree 2 k of nontrivial orbits in 
ty n (S h ) = described in Theorem 2.5. Besides, each matrix in X n fl S h forms itself a 

trivial orbit. The case n — 2k is treated in the same way. □ 
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6.2. The structure of the symmetric strata of the first (J5 n -action: an alternative 
approach. In §2.2 we have introduced the second (J5 n -action as the one induced by the 
first l2» n -action on the quotient of T n (F 2 ) modulo the subspace T n of the invariants of the 
first <5 n -action. Now we apply the same construction in the conjugate space. That is, 
we consider the action induced by the conjugate to the first (5 n -action on the quotient of 
(T n (F2))* modulo the subspace T> n of the dual invariants of the first (S> n -action. One can 
suggest the following natural description of this induced action. 

Consider the linear map $ n : (T n (F2))* — > T n_1 (F2) such that the (i, j)th entry in the 
image equals the sum of the entries + 1), (i + and (i + l,j + 1) in the 

inverse image (as before, entry in the image can be considered as representing the 
submatrix M l i of the initial matrix). Lemma 3.1 implies immediately that ker$ n = V n , 
and we thus obtain the induced action of (5 n on T n_1 (F2). 

The following observation gives a clue to the structure of the symmetric strata of the 
first (J5 n -action. 

Lemma 6.6. The <3 n -action on T n_1 (F 2 ) induced by $ n coincides with the second En- 
action. 

Proof. Simple exercise in linear algebra. □ 

Therefore, the above construction provides an alternative description of the second 
<5 n -action. As an immediate corollary of this description we obtain the structure of the 
stratum T>^ at height (0, . . . , 0). 

Lemma 6.7. The orbits of the first (5 n -action in the stratum are isomorphic to the 
orbits of the conjugate to the second <3 n -action. 

Proof. Indeed, consider the conjugate mapping $* : (T n_1 (F 2 ))* — > T n (F 2 ). It is easy 
to check that the image of $* coincides with T>^. Moreover, ker$* = 0, and hence $* 
provides an isomorphism between (T n_1 (F2))* and X>„ . Now Lemma 6.6 implies that the 
first C5 n -action on T>^ is isomorphic to the conjugate to the second (5 n -action. □ 

We thus get the part of Theorem 2.2 concerning the structure of symmetric strata. 

Lemma 6.8. (i) Let n = 2k + 1 ^ 5, then each of 2 k+1 symmetric strata consists of one 
orbit of length 2 2fc2+fc_1 - £ A .2 fc2+A: ~ 1 , one orbit of length 2 2fc2 + fc_1 + £ fc 2 fc2+fc_1 - 2 fc , and 
2 k orbits of length 1, where Sk = —1 for k = At + 1 and £fc = 1 otherwise; 

(ii) Let n = 2k ^ 6, then each of 2 k symmetric strata consists of two orbits of length 
( 2 2fc(fc-i) _ 1 )2 fc " 1 and 2 k orbits of length 1. 

Proof. The additive group X n acts on T n (F2) by translations. Evidently, any such trans- 
lation takes T>n to a symmetric stratum and any symmetric stratum is the translation 
of by a vector in T n . Besides, the action of T n commutes with the first (5 n -action, 
and hence takes orbits to orbits. Therefore, the orbit structure of any symmetric stratum 
coincides with that of the stratum T>^. 

To find the lengths of the orbits we use Lemmas 6.7, 5.6, and 4.3. Since k = k, we 
immediately get 2 k orbits of length 1. The lengths of the other two orbits are obtained 
readily from Corollary 5.3 and Lemma 5.4. □ 
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§7. Final Remarks 

7.1. The technique of the previous paper [SSV] is substantially generalized in a forthcom- 
ing paper joint with A. Zelevinsky to a wide class of intersections of pairs of Schubert cells 
and double Schubert cells. This generalization is based on the chamber ansatz developed 
for the flag varieties and semisimple groups in [BFZ, BZ, FZ]. For each reduced decom- 
position of an element u in a classical Coxeter group of type A, D, E we define a group 
which acts by symplectic transvections on the Z-module generated by all chamber sets 
associated with the chosen decomposition. We prove that the F2-reduction of the above 
action counts the number of connected components in in the intersection B n B u of two 
real open Schubert cells in G/B taken in the split form. Analogous results are obtained 
for the intersections G u,v = BuB n B_vB_ in a semisimple simply laced group G. 

These results lead to the following general setup. Given a connected undirected graph T, 
consider the F2-vector space Vr generated by its vertices and define an F2-valued bilinear 
form (p,q) = ^2{PiQj'- i adjacent to j}. Every vertex 5 G V determines the symplectic 
transvection T§ : Vr — > Vr sending p to p — (p, 5)5. Let us also choose some subset B of 
vertices of V and define the group <&b generated by all T$, 5 G B. 

Problem. Find the number of orbits of the (ds-action on Vr- 

A substantial part of the results of the present paper are valid in this more general 
setup as well. The authors hope to solve the above Problem at least for the graphs (and 
their groups) arising from intersections of Schubert cells. Our preliminary considerations, 
supported by numerical evidence, suggest the following surprising conjecture. 

Conjecture. The number of connected components in the intersection of two open Schu- 
bert cells in SL n+ i(M)/B in relative position w equals 3 • 2 n_1 for any generic w and 
n ^ 5. 

Here genericity means that the graph of bounded chambers introduced implicitly in 
[SSV] and studied in detail in the forthcoming paper with A. Zelevinsky contains an 
induced subgraph isomorphic to the Dynkin diagram of Eq. 

7.2. Another very intriguing fact is that the group (25 n studied in the present paper is the 
F2-reduction of the monodromy group of the Verlinde algebra su(3) n introduced in [Zu] 
and studied recently in [GZV]. (The authors are obliged to S. Chmutov and S. M. Gusein- 
Zade for this observation.) Notice that the monodromy group of any su(m) n is interpreted 
in [GZV] as the monodromy group of the isolated singularity of a certain Newton poly- 
nomial expanded in elementary symmetric functions. This gives us a hope to both find 
a natural representation of Verlinde algebra su(3) n in the sections of some bundle over 
the intersection of open opposite Schubert cells and, more generally, to find a relation be- 
tween the topology of intersections of Schubert cells and singularity theory of symmetric 
polynomials. 
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